Minimally invasive surgical techniques are aimed at reducing the amount of extraneous tissue that is damaged during diagnostic or surgical procedures, thereby reducing patient recovery time, discomfort, and deleterious side effects. As a consequence, the average length of a hospital stay for standard surgery may be shortened significantly using minimally invasive surgical techniques. Also, patient recovery time, patient discomfort, surgical side effects, and time away from work may also be reduced with minimally invasive surgery.
A common form of minimally invasive surgery is endoscopy, and a common form of endoscopy is laparoscopy, which is minimally invasive inspection and surgery inside the abdominal cavity. In standard laparoscopic surgery, a patient's abdomen is insufflated with gas, and cannula sleeves are passed through small (approximately ½ inch or less) incisions to provide entry ports for laparoscopic instruments. Laparoscopic surgical instruments generally include a laparoscope or an endoscope for viewing the surgical field.
An endoscope can be calibrated prior to use. Calibration is the process of determining intrinsic and extrinsic parameters for an imaging device by projecting three-dimensional (3-D) points into an image. Intrinsic parameters involve the internal geometric and optical characteristics of the imaging device, such as focal lengths in x and y, principal point in x and y, skew and pixel aspect ratio, and distortions (often quantified by a few parameters describing the distortions such as radial and tangential distortions). Intrinsic parameters can be used to compensate for imaging errors, such as optical aberrations of the imaging device. Extrinsic parameters involve the 3-D position of the camera reference coordinate system relative to a certain world coordinate system (i.e., six degree of freedom pose). In general, calibration is essential for many advanced imaging systems, such as advanced computer vision, 3-D augmented reality, 3-D visualization applications, advanced user interfaces, and image-guided surgery.
A stereoscopic imaging device, such as a stereo endoscope, is typically aligned at some point prior to use. The alignment process involves adjusting the left and right stereo images horizontally and vertically so as to have zero horizontal and vertical disparity at a certain distance. Without alignment, a viewer's eyes cannot properly fuse the left and right images (especially if the vertical disparity is large). Exemplary alignment methods and systems are described in commonly owned U.S. Pat. No. 7,277,120 (filed Mar. 7, 2004), which is hereby incorporated by reference. Calibration parameters for the two imaging paths of a stereo imaging device can provide parameters (horizontal and vertical offsets) of the alignment process.
Typical calibration methods involve imaging a calibration target. A calibration target typically has multiple features having known target relative coordinates. An image of the calibration target is processed so as to determine a collection of image coordinates associated with at least some of the target features. Known calibration methods can be used to process the collection of associated coordinates so as to generate calibration parameters, both extrinsic and intrinsic. (For exemplary methods, see Z. Zhang, “A flexible new technique for camera calibration,” IEEE trans. Pattern Analysis and Machine Intelligence, 2000, volume 22, number 11, pages 1330-1334; and Janne Heikkila and Olli Silven, “A Four-step Camera Calibration Procedure with Implicit Image Correction,” available at url <www.vision.caltech.edu/bouguetj/calib_doc/papers/heikkila97.pdf>, which are both hereby incorporated by reference.) Another method is implemented in a Matlab toolbox by Jean-Yves Bouguet (available at url <www.vision.caltech.edu/bouguetj/calib_doc/index.html>), which is a slightly modified version of the method described in the above-listed Zhang reference.
Calibration targets can be 3-D, two-dimensional (2-D), or one-dimensional (1-D). A 2-D target and related method(s) have a good balance of accuracy and convenience and are preferred in many applications. Calibration using planar targets requires multiple images of the target at different orientations so that the features being imaged have coordinates in three dimensions in any possible reference coordinate system, which is typically required by the matrix operations used to process the collection of associated coordinates. The exact poses of the target do not need to be known, since they can be estimated in the calibration process.
Existing methods used to extract obtain and process images of calibration targets suffer from a number of problems. For example, one calibration method involves imaging a checkerboard target pattern. The checkerboard pattern target must be properly positioned/oriented relative to the imaging device for multiple imaging directions. But properly placing the pattern as required by a calibration algorithm is not intuitive, and placement may therefore be difficult to guarantee. It can be especially difficult for non-technical persons to follow instructions directed to obtaining sufficiently different imaging directions. Additionally, since human hands are not very steady, holding the camera or target freehand typically induces motion blur. Some methods require manually designating corners of the pattern in the resulting images, such as the Matlab camera calibration tool box (see previous reference). As another example, the OpenCV computer vision library needs to have the number of grids of the pattern and requires that the full pattern be visible in an image.
There are some calibration methods that do not require manual designation. Some attach an attached optical tracking target to the calibration target to directly determine the 3-D information of the calibration target features (see Ramin Shahidi, Michael R. Bax, Calvin R. Maurer, Jr., Jeremy A. Johnson, Eric P. Wilkinson, Bai Wang, Jay B. West, Martin J. Citardi, Kim H. Wanwaring, and Rasool Khadem, “Implementation, Calibration and Accuracy Testing of an Image-Enhanced Endoscopy System,” In IEEE Transactions on Medical imaging, Vol. 21, No. 12, December 2002). Some add a few special features in the middle of the pattern that can be used to align the pattern with the image (see Christian Wengert, Mireille Reeff, Philippe C. Cattin, and Gabor Szekely, “Fully Automatic Endoscope Calibration for Intraoperative Use,” In Bildverarbeitung fur die Medizin Hamburg, 2006). However, this requires that the special pattern to be visible in an image, which eliminates the potential use of non-overlapping images of the target. As such, further improvements in calibration target design remain desirable, particularly target features that can be readily associated with their resulting images. More recently, some use self-identifying patterns for camera calibration (see Mark Fiala and Chang Shu, “Self-identifying patterns for plane-based camera calibration,” In Machine Vision and Applications (2008) 19:209-216). However, it does not provide a physical device/feature to interface with the imaging device to ensure that sufficient orientation variations have been captured and ease of use by non-technical users.
An endoscopic imaging system may also have its color balance (such as white balance) adjusted. In image processing, color balance involves the adjustment of the intensities of colors, typically the red, green, and blue primary colors. An important goal of this adjustment is to render specific colors correctly, particularly neutral colors. There are several aspects of image acquisition and display that result in a need for color balancing, including: that typical imaging device sensors do not match the sensors in the human eye, that the properties of the display medium impact the rendering of the color, and that the ambient conditions for the acquisition of the image may differ from the display viewing conditions. Color balance adjustment to keep neutral colors, such as white, neutral is sometimes called gray balance, neutral balance, or white balance, and this adjustment is a particularly important, if not dominant, element of color balancing.
It may also be advantageous to subject an endoscopic imaging system to diagnostic testing from time to time. A typical endoscopic imaging system includes a variety of components, such as imaging sensors, lens assemblies, etc., that may functionally degrade or fail over time. Where functional degradation that does not rise to an intolerable level has occurred, an endoscopic imaging system may continue to be used due to a lack of knowledge on the part of the user that any functional degradation has occurred. Such latent functional degradation may have significant detrimental consequences in a critical image-guided procedure, such as many minimally invasive surgeries.
While imaging device calibration, alignment, color balance, and diagnostic testing may be performed by using existing methods and devices, improved methods and assemblies for performing these tasks in a more convenient and efficient manner remain of interest. For example, methods and assemblies that can be conveniently used to perform these tasks all at once prior to a surgery would be of particular interest.